Strategies to improve retention in randomised trials: a Cochrane systematic review and meta-analysis

Objective: To quantify the effect of strategies to improve retention in randomised trials.<p></p> Design: Systematic review and meta-analysis.<p></p> Data sources Sources searched: MEDLINE, EMBASE, PsycINFO, DARE, CENTRAL, CINAHL, C2-SPECTR, ERIC, PreMEDLINE, Cochrane Methodology Register, Current Controlled Trials metaRegister, WHO trials platform, Society for Clinical Trials (SCT) conference proceedings and a survey of all UK clinical trial research units.<p></p> Review: methods Included trials were randomised evaluations of strategies to improve retention embedded within host randomised trials. The primary outcome was retention of trial participants. Data from trials were pooled using the fixed-effect model. Subgroup analyses were used to explore the heterogeneity and to determine whether there were any differences in effect by the type of strategy.<p></p> Results: 38 retention trials were identified. Six broad types of strategies were evaluated. Strategies that increased postal questionnaire responses were: adding, that is, giving a monetary incentive (RR 1.18; 95% CI 1.09 to 1.28) and higher valued incentives (RR 1.12; 95% CI 1.04 to 1.22). Offering a monetary incentive, that is, an incentive given on receipt of a completed questionnaire, also increased electronic questionnaire response (RR 1.25; 95% CI 1.14 to 1.38). The evidence for shorter questionnaires (RR 1.04; 95% CI 1.00 to 1.08) and questionnaires relevant to the disease/condition (RR 1.07; 95% CI 1.01 to 1.14) is less clear. On the basis of the results of single trials, the following strategies appeared effective at increasing questionnaire response: recorded delivery of questionnaires (RR 2.08; 95% CI 1.11 to 3.87); a ‘package’ of postal communication strategies (RR 1.43; 95% CI 1.22 to 1.67) and an open trial design (RR 1.37; 95% CI 1.16 to 1.63). There is no good evidence that the following strategies impact on trial response/retention: adding a non-monetary incentive (RR=1.00; 95% CI 0.98 to 1.02); offering a non-monetary incentive (RR=0.99; 95% CI 0.95 to 1.03); ‘enhanced’ letters (RR=1.01; 95% CI 0.97 to 1.05); monetary incentives compared with offering prize draw entry (RR=1.04; 95% CI 0.91 to 1.19); priority postal delivery (RR=1.02; 95% CI 0.95 to 1.09); behavioural motivational strategies (RR=1.08; 95% CI 0.93 to 1.24); additional reminders to participants (RR=1.03; 95% CI 0.99 to 1.06) and questionnaire question order (RR=1.00, 0.97 to 1.02). Also based on single trials, these strategies do not appear effective: a telephone survey compared with a monetary incentive plus questionnaire (RR=1.08; 95% CI 0.94 to 1.24); offering a charity donation (RR=1.02, 95% CI 0.78 to 1.32); sending sites reminders (RR=0.96; 95% CI 0.83 to 1.11); sending questionnaires early (RR=1.10; 95% CI 0.96 to 1.26); longer and clearer questionnaires (RR=1.01, 0.95 to 1.07) and participant case management by trial assistants (RR=1.00; 95% CI 0.97 to 1.04).<p></p> Conclusions: Most of the trials evaluated questionnaire response rather than ways to improve participants return to site for follow-up. Monetary incentives and offers of monetary incentives increase postal and electronic questionnaire response. Some strategies need further evaluation. Application of these results would depend on trial context and follow-up procedures.<p></p&gt

Use of strategies to improve retention in primary care randomised trials: a qualitative study with in-depth interviews

Objective To explore the strategies used to improve retention in primary care randomised trials.<p></p> Design Qualitative in-depth interviews and thematic analysis.<p></p> Participants 29 UK primary care chief and principal investigators, trial managers and research nurses.<p></p> Methods In-depth face-to-face interviews.<p></p> Results Primary care researchers use incentive and communication strategies to improve retention in trials, but were unsure of their effect. Small monetary incentives were used to increase response to postal questionnaires. Non-monetary incentives were used although there was scepticism about the impact of these on retention. Nurses routinely used telephone communication to encourage participants to return for trial follow-up. Trial managers used first class post, shorter questionnaires and improved questionnaire designs with the aim of improving questionnaire response. Interviewees thought an open trial design could lead to biased results and were negative about using behavioural strategies to improve retention. There was consensus among the interviewees that effective communication and rapport with participants, participant altruism, respect for participant's time, flexibility of trial personnel and appointment schedules and trial information improve retention. Interviewees noted particular challenges with retention in mental health trials and those involving teenagers.<p></p> Conclusions The findings of this qualitative study have allowed us to reflect on research practice around retention and highlight a gap between such practice and current evidence. Interviewees describe acting from experience without evidence from the literature, which supports the use of small monetary incentives to improve the questionnaire response. No such evidence exists for non-monetary incentives or first class post, use of which may need reconsideration. An exploration of barriers and facilitators to retention in other research contexts may be justified.<p></p&gt

Increasing follow-up questionnaire response rates in a randomized controlled trial of telehealth for depression: three embedded controlled studies

Background: Attrition is problematic in trials, and may be exacerbated in longer studies, telehealth trials and participants with depression – three features of The Healthlines Study. Advance notification, including a photograph and using action-oriented email subject lines might increase response rates, but require further investigation. We examined the effectiveness of these interventions in three embedded Healthlines studies. Methods: Based in different trial sites, participants with depression were alternately allocated to be pre-called or not ahead of the 8-month follow-up questionnaire (Study 1), randomized to receive a research team photograph or not with their 12-month questionnaire (Study 2), and randomized to receive an action-oriented (‘ACTION REQUIRED’) or standard (‘Questionnaire reminder’) 12-month email reminder (Study 3). Participants could complete online or postal questionnaires, and received up to five questionnaire reminders. The primary outcome was completion of the Patient Health Questionnaire (PHQ-9). Secondary outcome measures were the number of reminders and time to questionnaire completion. Results: Of a total of 609 Healthlines depression participants, 190, 251 and 231 participants were included in Studies 1–3 (intervention: 95, 126 and 115), respectively. Outcome completion was ≥90 % across studies, with no differences between trial arms (Study 1: OR 0.38, 95 % CI 0.07–2.10; Study 2: OR 0.84, 95 % CI 0.26–2.66; Study 3: OR 0.53 95 % CI 0.19–1.49). Pre-called participants were less likely to require a reminder (48.4 % vs 62.1 %, OR 0.41, 95 % CI 0.21–0.78), required fewer reminders (adjusted difference in means −0.67, 95 % CI −1.13 to −0.20), and completed follow-up quicker (median 8 vs 15 days, HR 1.35, 95 % CI 1.00–1.82) than control subjects. There were no significant between-group differences in Studies 2 or 3. Conclusions: Eventual response rates in this trial were high, with no further improvement from these interventions. While the photograph and email interventions were ineffective, pre-calling participants reduced time to completion. This strategy might be helpful when the timing of study completion is important. Researchers perceived a substantial benefit from the reduction in reminders with pre-calling, despite no overall decrease in net effort after accounting for pre-notification

What are the most important unanswered research questions in trial retention? A James Lind Alliance Priority Setting Partnership: the PRioRiTy II (Prioritising Retention in Randomised Trials) study

Background One of the top three research priorities for the UK clinical trial community is to address the gap in evidence-based approaches to improving participant retention in randomised trials. Despite this, there is little evidence supporting methods to improve retention. This paper reports the PRioRiTy II project, a Priority Setting Partnership (PSP) that identified and prioritised unanswered questions and uncertainties around trial retention in collaboration with key stakeholders. Methods This PSP was conducted in collaboration with the James Lind Alliance, a non-profit making initiative, to support key stakeholders (researchers, patients, and the public) in jointly identifying and agreeing on priority research questions. There were three stages. (1) First an initial online survey was conducted consisting of six open-ended questions about retention in randomised trials. Responses were coded into thematic groups to create a longlist of questions. The longlist of questions was checked against existing evidence to ensure that they had not been answered by existing research. (2) An interim stage involved a further online survey where stakeholders were asked to select questions of key importance from the longlist. (3) A face-to-face consensus meeting was held, where key stakeholder representatives agreed on an ordered list of 21 unanswered research questions for methods of improving retention in randomised trials. Results A total of 456 respondents yielded 2431 answers to six open-ended questions, from which 372 questions specifically about retention were identified. Further analysis included thematically grouping all data items within answers and merging questions in consultation with the Steering Group. This produced 27 questions for further rating during the interim survey. The top 21 questions from the interim online survey were brought to a face-to-face consensus meeting in which key stakeholder representatives prioritised the order. The ‘Top 10’ of these are reported in this paper. The number one ranked question was ’What motivates a participant’s decision to complete a clinical trial?’ The entire list will be available at www.priorityresearch.ie. Conclusion The Top 10 list can inform the direction of future research on trial methods and be used by funders to guide projects aiming to address and improve retention in randomised trials

Systematic techniques for assisting recruitment to trials (START): study protocol for embedded, randomized controlled trials

BACKGROUND: Randomized controlled trials play a central role in evidence-based practice, but recruitment of participants, and retention of them once in the trial, is challenging. Moreover, there is a dearth of evidence that research teams can use to inform the development of their recruitment and retention strategies. As with other healthcare initiatives, the fairest test of the effectiveness of a recruitment strategy is a trial comparing alternatives, which for recruitment would mean embedding a recruitment trial within an ongoing host trial. Systematic reviews indicate that such studies are rare. Embedded trials are largely delivered in an ad hoc way, with interventions almost always developed in isolation and tested in the context of a single host trial, limiting their ability to contribute to a body of evidence with regard to a single recruitment intervention and to researchers working in different contexts. METHODS/DESIGN: The Systematic Techniques for Assisting Recruitment to Trials (START) program is funded by the United Kingdom Medical Research Council (MRC) Methodology Research Programme to support the routine adoption of embedded trials to test standardized recruitment interventions across ongoing host trials. To achieve this aim, the program involves three interrelated work packages: (1) methodology - to develop guidelines for the design, analysis and reporting of embedded recruitment studies; (2) interventions - to develop effective and useful recruitment interventions; and (3) implementation - to recruit host trials and test interventions through embedded studies. DISCUSSION: Successful completion of the START program will provide a model for a platform for the wider trials community to use to evaluate recruitment interventions or, potentially, other types of intervention linked to trial conduct. It will also increase the evidence base for two types of recruitment intervention. TRIAL REGISTRATION: The START protocol covers the methodology for embedded trials. Each embedded trial is registered separately or as a substudy of the host trial

A study update newsletter or Post-it® note did not increase postal questionnaire response rates in a falls prevention trial: an embedded randomised factorial trial

Background: Participants not returning data collection questionnaires is a problem for many randomised controlled trials. The resultant loss of data leads to a reduction in statistical power and can result in bias. The aim of this study was to assess whether the use of a study update newsletter and/or a handwritten or printed Post-it® note sticker increased postal questionnaire response rates for participants of a randomised controlled trial. Method: This study was a factorial trial embedded within a host trial of a falls-prevention intervention among men and women aged ≥65 years under podiatric care. Participants were randomised into one of six groups: newsletter plus handwritten Post-it®; newsletter plus printed Post-it®; newsletter only; handwritten Post-it® only; printed Post-it® only; or no newsletter or Post-it®. The results were combined with those from previous embedded randomised controlled trials in meta-analyses. Results: The overall 12-month response rate was 803/826 (97.2%) (newsletter 95.1%, no newsletter 99.3%, printed Post-it® 97.5%, handwritten Post-it® 97.1%, no Post-it® 97.1%). The study update newsletter had a detrimental effect on response rates (adjusted odds ratio 0.14, 95% CI 0.04 to 0.48, p<0.01) and time to return the questionnaire (adjusted hazard ratio 0.86, 95% CI 0.75 to 0.99, p=0.04). No other statistically significant differences were observed between the intervention groups on response rates, time to response, and the need for a reminder. Conclusions: Post-it® notes have been shown to be ineffective in three embedded trials, whereas the evidence for newsletter reminders is still uncertain. Keywords Randomised controlled trial; randomisation; embedded trial; newsletter; Post-it® note; response rat

The Early Positive Approaches to Support (E-PAtS) study: study protocol for a feasibility cluster randomised controlled trial of a group programme (E-PAtS) for family caregivers of young children with intellectual disability

Background: Children with intellectual disability have an IQ < 70, associated deficits in adaptive skills and are at increased risk of having clinically concerning levels of behaviour problems. In addition, parents of children with intellectual disability are likely to report high levels of mental health and other psychological problems. The Early Positive Approaches to Support (E-PAtS) programme for family caregivers of young children (5 years and under) with intellectual and developmental disabilities is a group-based intervention which aims to enhance parental psychosocial wellbeing and service access and support positive development for children. The aim of this study is to assess the feasibility of delivering E-PAtS to family caregivers of children with intellectual disability by community parenting support service provider organisations. The study will inform a potential, definitive RCT of the effectiveness and cost-effectiveness of E-PAtS. Methods: This study is a feasibility cluster randomised controlled trial, with embedded process evaluation. Up to 2 family caregivers will be recruited from 64 families with a child (18 months to 5 years) with intellectual disability at research sites in the UK. Participating families will be allocated to intervention: control on a 1:1 basis; intervention families will be offered the E-PAtS programme immediately, continuing to receive usual practice, and control participants will be offered the opportunity to attend the E-PAtS programme at the end of the follow-up period and will continue to receive usual practice. Data will be collected at baseline, 3 months post-randomisation and 12 months post-randomisation. The primary aim is to assess feasibility via the assessment of: recruitment of service provider organisations; participant recruitment; randomisation; retention; intervention adherence; intervention fidelity and the views of participants, intervention facilitators and service provider organisations regarding intervention delivery and study processes. The secondary aim is preliminary evaluation of a range of established outcome measures for individual family members, subsystem relationships and overall family functioning, plus additional health economic outcomes for inclusion in a future definitive trial. Discussion: The results of this study will inform a potential future definitive trial, to evaluate the effectiveness and cost-effectiveness of the E-PAtS intervention to improve parental psychosocial wellbeing. Such a trial would have significant scientific impact internationally in the intellectual disability field

Scaphoid Waist Internal Fixation for Fractures Trial (SWIFFT) protocol : a pragmatic multi-centre randomised controlled trial of cast treatment versus surgical fixation for the treatment of bi-cortical, minimally displaced fractures of the scaphoid waist in adults

BACKGROUND: A scaphoid fracture is the most common type of carpal fracture affecting young active people. The optimal management of this fracture is uncertain. When treated with a cast, 88 to 90 % of these fractures unite; however, for the remaining 10-12 % the non-union almost invariably leads to arthritis. The alternative is surgery to fix the scaphoid with a screw at the outset. METHODS/DESIGN: We will conduct a randomised controlled trial (RCT) of 438 adult patients with a "clear" and "bicortical" scaphoid waist fracture on plain radiographs to evaluate the clinical effectiveness and cost-effectiveness of plaster cast treatment (with fixation of those that fail to unite) versus early surgical fixation. The plaster cast treatment will be immobilisation in a below elbow cast for 6 to 10 weeks followed by mobilisation. If non-union is confirmed on plain radiographs and/or Computerised Tomogram at 6 to 12 weeks, then urgent surgical fixation will be performed. This is being compared with immediate surgical fixation with surgeons using their preferred technique and implant. These treatments will be undertaken in trauma units across the United Kingdom. The primary outcome and end-point will be the Patient Rated Wrist Evaluation (a patient self-reported assessment of wrist pain and function) at 52 weeks and also measured at 6, 12, 26 weeks and 5 years. Secondary outcomes include an assessment of radiological union of the fracture; quality of life; recovery of wrist range and strength; and complications. We will also qualitatively investigate patient experiences of their treatment. DISCUSSION: Scaphoid fractures are an important public health problem as they predominantly affect young active individuals in the more productive working years of their lives. Non-union, if untreated, can lead to arthritis which can disable patients at a very young age. There is a rapidly increasing trend for immediate surgical fixation of these fractures but there is insufficient evidence from existing RCTs to support this. The SWIFFT Trial is a rigorously designed and adequately powered study which aims to contribute to the evidence-base to inform clinical decisions for the treatment of this common fracture in adults. TRIAL REGISTRATION: The trial is registered with the International Standard Randomised Controlled Trial Register ( ISRCTN67901257 ). Date registration assigned was 13/02/2013

Retaining Participants in Community-Based Health Research: A Case Example on Standardized Planning and Reporting

Background Effective strategies for participant retention are critical in health research to ensure validity, generalizability and efficient use of resources. Yet standardized guidelines for planning and reporting on retention efforts have been lacking. As with randomized controlled trial (RCT) and systematic review (SR) protocols, retention protocols are an opportunity to improve transparency and rigor. An RCT being conducted in British Columbia (BC), Canada provides a case example for developing a priori retention frameworks for use in protocol planning and reporting. Methods The BC Healthy Connections Project RCT is examining the effectiveness of a nurse home-visiting program in improving child and maternal outcomes compared with existing services. Participants (N&thinsp;=&thinsp;739) were girls and young women preparing to parent for the first time and experiencing socioeconomic disadvantage. Quantitative data were collected upon trial entry during pregnancy and during five follow-up interviews until participants’ children reached age 2 years. A framework was developed to guide retention of this study population throughout the RCT. We reviewed relevant literature and mapped essential retention activities across the study planning, recruitment and maintenance phases. Interview completion rates were tracked. Results Results from 3302 follow-up interviews (in-person/telephone) conducted over 4 years indicate high completion rates: 90% (n&thinsp;=&thinsp;667) at 34&thinsp;weeks gestation; and 91% (n&thinsp;=&thinsp;676), 85% (n&thinsp;=&thinsp;626), 80% (n&thinsp;=&thinsp;594) and 83% (n&thinsp;=&thinsp;613) at 2, 10, 18 and 24 months postpartum, respectively. Almost all participants (99%, n&thinsp;=&thinsp;732) provided ongoing consent to access administrative health data. These results provide preliminary data on the success of the framework. Conclusions Our retention results are encouraging given that participants were experiencing considerable socioeconomic disadvantage. Standardized retention planning and reporting may therefore be feasible for health research in general, using the framework we have developed. Use of standardized retention protocols should be encouraged in research to promote consistency across diverse studies, as now happens with RCT and SR protocols. Beyond this, successful retention approaches may help inform health policy-makers and practitioners who also need to better reach, engage and retain underserved populations